Toward a Definitive Compressibility Measure for Repetitive Sequences

While the $k$ th order empirical entropy is an accepted measure of compressibility individual sequences on classical text collections, it useful only for small values and thus fails to capture repetitive sequences. In absence established way quantifying latter, ad-hoc measures like size notation="LaTeX">$z$ Lempel–Ziv parse are frequently used estimate repetitiveness. The notation="LaTeX">$b \le z$ smallest bidirectional macro scheme captures better what can be achieved via copy-paste processes, though NP-complete compute, not monotone upon appending symbols. Recently, a more principled measure, notation="LaTeX">$\gamma $ string attractor , was introduced. b$ lower-bounds all previous relevant ones, while length- notation="LaTeX">$n$ strings represented efficiently indexed within space notation="LaTeX">$O\left({\gamma \log \frac {n}{\gamma }}\right)$ which also upper-bounds many measures, including . Although arguably repetitiveness than notation="LaTeX">$b$ compute monotone, unknown if one represent in notation="LaTeX">$o(\gamma n)$ space. this paper, we study even smaller notation="LaTeX">$\delta \gamma computed linear time, allows encoding every string notation="LaTeX">$O\left({\delta {n}{\delta because notation="LaTeX">$z = O\left({\delta We argue that strings. Concretely, show (1) strictly by up logarithmic factor; (2) there families needing notation="LaTeX">$\Omega \left({\delta encoded, so optimal ; (3) build run-length context-free grammars whereas (non-run-length) grammar notation="LaTeX">$\Theta (\log n/\log times larger; (4) space, but offer logarithmic-time access its symbols, computation substring fingerprints, efficient searches pattern occurrences. further refine above results account alphabet notation="LaTeX">$\sigma string, showing {n\log \sigma }{\delta n}}\right)$ necessary sufficient support access, fingerprinting, matching queries.